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ABSTRACT 

This paper outlines the rationale and basic design 
for the development of a state-based network for improving indicators 
of precollege science and ::Athematics education. The historical 
background for improving the indicators of the federal and state 
levels are described. Five elemented of a conceptual framework for the 
network are discussed. These include: (1) "Defining and Identifying 
Indicators"; (2) "Current Goals for Improving Science and Mathematics 
Education"; (3) "Using Existing Frameworks and Model Indicators"; (4) 
"Building on the CCSSO Model of Educational Indicators"; and (5) 
"Responding to Needs of States." Three phases of this project are 
discussed. Six categories of the indicators of science and 
mathematics are listed and described, including student outcomes, 
instructional time, curriculum content, school conditions, teacher 
quality, ana resources. A total of 15 references are listed. (YP) 
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SCIENCE AND ^IATHEMATICS INDICATORS: 
OONCEPIUAL FRAMEWORK FOR A STATE-BASED NETVXDRK 



This paper outlines the rationale and basic design for the development 
of a state-based network for iirproving indicators of precollege science 
and mathematics education. Uie project is a new undertaking for the 
council of C3iief state School Officers (CCSSO) throu^ the si^port of the 
National Science Foundation. It is being planned as a major addition to 
the activities of the Ccuncil's State Educational Assessment Center, 

The goal of the network project is to work with state education 
agencies to develop better indicators of the characteristics and quality 
of science and mathematics in elementary and secondary schools. Inproving 
state indicators thrau<^ the efforts of CCSSO and the states would have 
two major benefits: a) iitpruve the quality and usefulness of information 
at the state level vdiere educational decisions are gaining in inportance, 
and b) develop ccaiparable state data, and thus, provide a more accurate, 
conprehensive national database on science and mathematics education. 
The Education Reforms and Science and Mathematics 

The initial inpetus for iirproving educational indicators at national 
and state levels came from recommendations and reports of several national 
commissions and study groi^xs, such as the National Commission on 
Excellence in Education (1983) , the National Science Board Commission on 



Precollege Education in Mathematics, Science ard Technology (1983); the 
•Twentieth Century Fund (1983) ; and the Education Commission of the States' 
Task Force on Education for Economic Growth (1983) . These reports 
described serious inadequacies in the quality of elementary and secondary 
education. A particular concern of the reports was that most U.S. high 
school graduates leave school with poor preparation in science and 
mathematics, vdiether for the job market or for a college education. 
Specific prc4)lems in schools that were cited as contributing to poor 
science and mathematics education were teacher shortages, quality of 
teacher preparation, low standards and e^q^ectations for learning, and 
inadequate curricula. 

Over the past three years, the federal and state governments have 
responded to the reconroendations for change in education with significant 
reform legislation, and private foundations and business and industiy have 
also increased their involvement in elementary and secondary education. 
New federal legislation provided increased funding for training science 
and mathematics teachers and for curriculum develc^iment, itrproving 
teaching materials, and research on instruction in science and 
mathematics. However, the reforms that have itvost directly addressed the 
quantity and quality of education in science and mathematics have been 
initiated at the state level. Many states have increased science and 
mathematics course requirements for high school graduation, provided 
incentives to attract and retain teachers, and r^iewed and strengthened 
curricula at the elementary and secondary levels. 



InorovirCT Infonration on Science and l»iathematics Education 

The reccanmendations for change in elementary and secondary education 
were accoitpanied by recf/iranendations for iiiprovement in the quality and 
availability of information on the performance of the educational system. 
For example, the r^rt of the National Science Board Commission, 
Educatincf Americans for the 21st Century ^ noted major problems with the 
quality and usefulness of data on student outccares, characteristics of 
teachers, and school conditions. The Commission cited the need for better 
measures of education quality .ind methods of collecting and aggregating 
data that would allow regular monitoring at national, state, and local 
levels. 

As the varicus reforms directed toward education quality progressed, 
federal and state governments have becane more aware of the inportance of 
having useful information on the status of education and the effects of 
educational inprovement initiatives. The National Science Foundation 
(NSF) has been inproving its capacity for monitoring the condition of 
education in science, mathematics, and technology. The shortcomings of 
existing data and indicators were outlined in the ch^>ter on Science 
Education in the 1985 edition of NSF's biennual r^rt, science 
Indicators , since 1984, the Science and Engineering Education Directorate 
of NSF has initiated a series of projects that will provide new 
information on the status of science and mathematics in our sdiools, add 
to existing sources of information, and advise NSF and others on 
directions for revising current systems for collecting and analyzing data. 

The states have placed new eirphasis on educational indicators in 
conjunction with the series of reforms being inplemented at that level. 
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Many state education agencies have developed assessntent tests in various 
subject areas and at a number of grade levels, in addition to state 
coopetency tests for prcanotion or graduation (CCSSO, 1984) . State efforts 
to increase pay and incentives for teachers have been accoatpanied by tests 
of teacher ccnpetencies and skills. Currently, 35 states require teachers 
to pass a written test for certification, and some states have developed 
new methods of assessing teaching performance in the classroom (ECS, 
1985) . However, with the current education reform activity, states ar^ 
not only developing new tests, th^ are looking at their entire systems of 
inf ormation-gatiiering to determine how well they serve the needs of 
program managers and policy-makers. 
ImprovincT State Indicators 

Ihe state-based network to develqp science and mathematics indicators 
is being initiated because of the interest and ccanmitment of the states, 
and NSF, to inproving educational indicators and systems for monitoring 
education quality. In 1984, the Counci-l of C3iief State School Officers 
made a fundamental shift in its policy on the role of states in 
educational assessment and evcaiaation. In a major policy paper, the CCSSO 
affirmed the responsibility of states to ensure that the data on education 
collected, by states were of hi^ technical quality and useful consistency 
frcan state to state (CCSSO, 1984) . similarly, the National Governors' 
Association has called for "report cards" on states' efforts to make 
educational iitproveraents in seven areas (1985) . In a recent report, the 
governors recommended steps to iroprove teaching and provide better 
infomation on teachers (NGA, 1986) . By building on existing efforts by 
states to iroprove data on education, this project provides an excellent 
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opportuniti' to iirprcvs the cciiprehensiveness, utility, conparabiiity, and 
technical quality of, state indicators of precollege education in science 
and mathematics* 

Elements of a Conceptual Framework for the State-Based Network 

The project plan for working with states to iitprove indicatc-rs of 

science and mathematics is based on a conc^)tual framework. The framework 

provides linkages between current work on indicators and the states' role 

in science and mathematics education. 

Five elements ccanoprise the conceptual framework for the state-based 

network: 

DefinincT and Identi fying Indicators , The objective of irtproving 
state indicators of science and mathematics is not to increase the amount 
of information that is collected and analyzed by states, but rather to 
eir^iiasize the use of data and information by policy-makers, managers, ard 
educators. In another field, economic indicators such as the unenployment 
rate or the gross national product reflect only a very small, portion of 
the data on the American econatiy that are collected by the federal 
government, but the indicators are widely known and used. 

An "indicator*' has been defined by a ccatimittee of the National Academy 
of Sciences as follows: 

. . .a measure that conveys a general iirpression of the state or nature 
of the structure or system being examined. While it is not 
necessarily a precise statement, it gives sufficient indication of a 
condition conceming the system of interest to be of use in 
fonnulating policy. . .Optimally, an indicator combines information on 
conceptually related variables, so that the number of indicators 
needed to describe the system of concern can be kept reasonably 
small. (Paizen and Jonr-s, 1985, p. 27) 

With respect to states, educational indicators may coirprise a selected 

portion of the data and statistics gathered and analyzed by an education 

agency. 
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In dsvalcping a system of educational indicators, the local, state, or 
federal "policy context" plays a major role in the selection and 
inplementation of indicators (Oakes, 1986) . A central idea of this 
project is to organize the identification and iirprovement of indicators 
around a network of three r^resentatives frcan each state education 
agency, ihe CCSSO and the network will work on "brokering" irtprovements 
with states, and building on the systems of data collection that are in 
place. 

The process of identifying and developing science and mathematics 
indicators throiw^ the state network will involve three steps: a) 
collecting and synthesizing information on state data collectif 
practices, and coiparing the types of infonnation available with model, or 
"ideal", indicators, b) working with states to develc^) strategies for 
inproving indicators, and c) assisting individual states with iirplementing 
inprovements in their systems of indicators. 

2* Current Goals for anprovina Science and Mathematics Education . 
Research or analysis of a system typically has an underlying theory or set 
of assunptions about how the system should function. In our society, 
there are a variety of well-known values and goals for elenentary and 
secondary education. The recent national canmission r^rts have been 
quite e)q)licit about the desired directions for itiprovement in science and 
mathematics (National Science Board, 1983; Education Commission of the 
States, 1983; Twentieth Century Fund, 1983; Raizen and Jones, 1985; 
Carnegie Forum on Education and the Econoniy, 1986) . Several common goals 
for inproving the quality of science and mathematics education in our 
schools are found in these reports, and many state education reforms have 
been framed with these goals in mind: 



o Advancim Science and Mathematics Literacy of all Students , 

Science and mathematics are iirportant core areas of learning for 
all students, regardless of their career intentions. Students 
should leave school being literate in science and mathematics and 
have the pr'^>aration for renewing their knowledge of science 
throfughout life, 

o Education and the Econoitiy , 

Schools need to increase the level of student learning in science 
and mathematics for the itoerican econotY to remain CQirpetitive. 
•me growing central role of technology in itost areas of the 
econanir demand requisite knowledge in science and mathematics. 

o Assessing Leciming , 

Student achieveapaent in science and mathematics should be assessed 
with direct measurejoent of the skills, processes, and content 
learned in school, and not sinply by counting the grade level or 
number of ccurses cccpleted by students. 

o ijnDrovincf Teaching ( ;)iyi|-i,i- y - 

Inproving the preparation of new teachers and attracting and 
retaining hi^ily qualified pec5)le to teach science and 
mathematics are necessary to advance science and mathematics 
education in the U.S. 

Indicators developed through the project should reflect these current 
directions for inproving science and mathematics education. 

3* Using Existing Fr ameworks and Model Indicators , ihe work with 
states to develop and inoprove mathematics and science indicators will 
benefit frcapci findings and reccnmendations frcm several recent studies and 
projects. 

A ccanmittee of the the National Academy of Sciences/tTational Research 
council is develcpLng indicators that are needed to adequateJ,y monitor and 
assess the quality of education in science and mathematics in elementary 
and secondary schools. This study of model indicators follows an earlier 
analysis of the quality and usefulness of existing science and matherratics 
indicators that are available to education policy-makers, program 
managers, and educators (Raizen and Jones, 1985) . 
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Ihe Pand Corporation is conducting a study of alternative prototypes 
for a national monitoring system in mathematics, science, ard technology 
education, ihe project involves a review of research on the relationship 
of (iharacteristics of schools, classrocaans, and teachers to student 
outccanes in science and mathematics, identification of critical variables 
in a ccaiprehensive model .of the education system, and develc^ment of 
several models for selecting indicators for a national monitorii^ system. 

A mathematics monitoring center has been developed at the University 
of Wisconsin-Madison, which is analyzing the effects of education reform 
efforts on mathematics instruction and student learning. To acccmplish 
tliis task, the center is developing a model to collect and analyze data 
that explain the performance of students in mathematics and to identify 
inportant educational indicators for trackii^ trends in elementary and 
secondary mathematics. 

The recoiimendations for science and mathematics indicators from these 
three projects will inform the selection of "ideal" indicators for 
analyzing current state data and indicators. In addition, findings and 
recanmendations from other recent national studies on education will 
contribute to the selection of "ideal" indicators, such as the Carnegie 
Forum on Education and the Econcnty (1986) , the National Governors 
Association (1986) , and the National Science Board ccsnmlssion on 
Precollege Education in Mathematics, Science and Itechnology (1983) . 
Several national surveys and studies have used data items that are likely 
to be useful for developing state indicators, including the 1985-86 
National Survey of Science and Mathematics Education, the 1985-86 National 
Assessments of Educational Progress in Mathematics and Science, and the 
Hi^ School and B^ond surveys. 
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4. Rnildinq n n the CCSSQ Model of Educational Indicators . 
In October 1985, the Committee on CoorxiinatiJig Educational Information and 
Research (CEIR) of the CCSSO approved a model for state education 
indicators. Ihe CCSSO model has three ccaiponents: a) indicators of 
educational outcomes, b) indicators of school policies and practices, and 
c) indicators of contextual factors. 

CEIR recommended a core set of indicators for the model, and these 
recommendaticans provided an outline for the OCSSO program to develop a 
system of catparable state data on education. Three criteria were 
established by CEIR for selecting the core indicators: importance/use by 
states, technical quality, c.nd feasibility; and it was decided that the 
three criteria should be applied in this ortJer. 

Since the adcption of the CEER r^rt, CCSSO has begun putting the 
core set of indicators into place, ihe development of science and 
mathematics indicators in this project will follow the CCSSO model; the 
categories of indicators will be consistent with the model, and specific 
measures will be reccmmended that meet the review criteria. Thus, the 
science and mathematics indicators project will serve as a means of 
implementing the model approved by the Chiefs, but the work will be 
concerned with developing indicators and measures that are specific to 
science and mathematics. 

Fortunately, a good deal of information about state indicators and 
data is already available. E>dsting sources of information include the 
UdA study of state assessment tests (Burstein, 1985) , the recent ECS 
r^rts on graduation requirements (1985) and on state teacher policies 
(1985) , surveys of state competency tests and assessment tests (ECS, 1985; 
CCSSO, 1984) , a CCSSO study of states' uses of automated information 



systems (1982) , a recent study by BIS of state educational standards 
(Goertz, 1986) , and a forthccmdng stud^^ by the National Research Council 
on statistical models for teacher simply and demand. 

5- Responding. to Needs of States . In its irtdicatc^rs pTOgrcm, the 
OCSSO is working to develop states' capacity to assess educational 
oatccanes by improving the quality and usefulness of inforroation, not by 
increasing local and stat:e regulation and data burden. The state-based 
network to develop science and mathematics indicators will be designed 
with the same view toward balancing the need for a more conplete picture 
of education with the constraint of selecting the most useful, valid, and 
feasible indicators of school policies and practices and student outcomes. 

Reviewers of the CCSSO proposal for this project noted that it is very 
iiiportant that specific information be obtained about current state 
definitions and practices in collectijng information on science and 
mathematics education. Specific di.'^ferences between the states need to be 
coipiled and analyzed in order to assist states in inproving science and 
mathematics indicators and developing carparable state data. For exaitple, 
information on the skills and knowlec^e in science that are expected in 
specific state tests for teadiers is likely to be more useful to states 
than sinply learning hew mai^ states have teacher carpetency tests in 
science. 

Ohe Cduix:il of Chief State School Officers has demonstrated that very 
specific, useful information on state data collection definitions and 
procedures can be cdDtained in an efficient manner, ihe Education Data 
Improvement Project that is currently being conducted for the U.S. 
D^>artment of Education involved analysis of the core^ta program, with 
each state being asked to verify and elaborate the definitions, 
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instrumants, and cjperational procedures tliey use for each data element. 
Successive iterations of verification were used to clarify arr^^ multiple 
interpretations that were possible in the state's definitions and 
procedures* 

In the new project, the same type of iterative process will be used to 
conpile a set of hi^y specific profiles of how each state 
operationalizes and uses a number of indicators of education in science 
and nathematicsc The ccaipleted "inventory" of state definitions and 
practices will provide the basis for assessing the need for revising and 
inproving state indicators • 
HTiTTaia T:f^ag es of the Project 

The project will be carried out throu*^ three najor st^ or phases: 
1. In the first year of the project, an inventory will be conducted 
with states on their current definitions and procedures for collecting 
information with selected types of indicators, A state network ccnprised 
of 3 r^resentatives frcm eadi state education agency will coordinate the 
inventori' process in eadi state. A shuttle process \dJ.l be used to report 
and verify the information with CCSSO, with the first round of the shuttle 
to rqport on the activities of the states with each type of indicator and 
the second round to provide details on definitions and specific 
procedures. Hie plans for the project will be carefully reviewed by a 
IS-member advisory board. Ihe project staff at the CCSSO will conpile the 
information provided by the states, and ccaipare the recent state 
definitions and procedures with ideal indicators of science and 
mathematics education. Ihe results; of the analysis will demonstrate the 
types of indicators tliat are likely to lead to conparable state 
information and the indicators that will require more work by some states. 
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2* In the second year, the state neta/crk participants and CCSSO staff 
will work together to establish priorities and strategies for iirproving 
indicators in science and mathematics education and moving towards a 
cross-state database on science and mathematics. A state-by--state plan 
will be developed for revising or i:pgrading the collection of data on 
science and mathematics indicators, based on the findings from the 
inventory of state practices and the established priorities. 

3 . Wbrk will begin on iitplementing state plans for indicators in the 
third year of the project. Ihe CCSSO staff will provide technical 
assistance to each state and coordinate the inplementation of an inproved 
system for data indicators in mathematics and sci^ce education. The 
system of indicators will be characterized by high quality, utility, 
cross-state cccparability, emd the capacity for aggregation .of data and 
indicators on science and mathematics education across states. 

Design for Science and Mathenatics Indicators 

Initially, six categories of indicators of science and mathematics 
will be eocamined: l) student outccmes, 2) ins^-ructional tirne, 3) 
curriculum content, 4) school conditions, 5) teacher quality, and 6) 
resources. 

For each type of indicator, information will be collected and verified 
on current state practices and definitions (including level of aggregation 
and .T^thod of collecting data) , plans for develcping indicators, and uses 
of indicators. This draft list of types of indicators will be revised as 
the project proceeds; it is provided here to outline the structure of the 
indicators frainework. 
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student Outcomes 

Following indicators by student gender and race/ethnicity: 

o Student achievement based on assessment of learning in science 
and mathoDoatics, in terms of specified, expected concepts, 
processes, and skills; assessment at elementary, intermediate, 
and secondary levels; 

o College majors in science and mathematics (sophorore year) ; 

o Rate of students entering careers requiring science and 
mathematics pr^jaration. 

Instructional Time 

o Time per week on science and mathematics in elementary grades; 

o Number of science and mathematics courses during grades 7 - 12 ; 

o Total enrollment by course title in science and mathematics 
grades 7 - 12; 

o Hours of hooaework per week in science and in mathematics. 
Curriculum Content 

o CJiaracteristics of curriculxmi frameworks for science and 
mathematics; 

o Ccnparison of curriculum frameworks with levels of science and 
mathanatics curricula; 

o Breadth and depth of coverage of content topics, based on 
informaticai fcon teachers. 

School Conditions 

Following indicators by characteristics of students in school: 

o Number of students per science and mathematics teacher; 

o Number of classrocnv/laboratory aides per science and mathematics 
teacher; 

o Hi(^ school course offerings in science and mathematics; 
o Teacher perc^ions of working conditions. 



Teacher Quality 



o Eleroents of state model to project teacher supply arid demand in 
science and itathematics ; 

o Characteristics of "new hires" in science and mathemtics, 
including status after two years. 



Following indicators by teaching assignnient in science or mathematics 
(subject, grade level) : 

o Assessment of teachers' knowledge of subject content and pedagogy 
in subject; 

o r*)served teaching performance in subject area; 

o Education and stqpervised teaching e>5)erience in subject area; 

o State certification requirements in science ax^ mathematics; 

o Years of e5q)erience, total, and in teadiiiij assignment; 

o Professicaial developoient throu^ formal courses and through 
in-service and professional activities. 

Resources 

o Average teadier salaries in mathematics and science by colleae 
major; 

o Ratio of teacher salary of science and mathematics majors to 
salary of non-teadiir^ science and mathematics majors in state; 

o Availability and use of equdpraent, instructional materials, texts, 
and laboratory facilities. 
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